Self-monitoring accuracy does not increase throughout undergraduate medical education

Context: Accurate self-assessment of one's performace on a moment-by-moment basis (ie, accurate self-monitoring) is vital for the self-regulation of practising physicians and indeed for the effective regulation of self-directed learning during medical education. However, little is currently known about the functioning of self-monitoring and its co-development with medical knowledge across medical education. This study is the first to simultaneously investigate a number of relevant aspects and measures that have so far been studied separately: different measures of self-monitoring for a broad area of medical knowledge across 10 different performance levels. Methods: This study assessed the self-monitoring accuracy of medical students (n = 3145) across 10 semesters. Data collected during the administration of the formative Berlin Progress Test Medicine (PTM) were analysed. The PTM comprises 200 multiple-choice questions covering all major medical disciplines and organ systems. A self-report indicator (ie, confidence) and two behavioural indicators of self-monitoring accuracy (ie, response time and the likelihood of changing an initial answer to a correct rather than an incorrect item) were examined for their development over semesters. Results: Analyses of more than 390 000 observations (of approximately 250 students per semester) showed that confidence was higher for correctly than for incorrectly answered items and that 86% of items answered with high confidence were indeed correct. Response time and the likelihood of the initial answer being changed were higher when the initial answer was incorrect than when it was correct. Contrary to expectations, no differences in self-monitoring accuracy were observed across semesters. Conclusions: Convergent evidence from different measures of self-monitoring suggests that medical students self-monitor their knowledge on a question-by-question basis well, although not perfectly, and to the same degree as has been found in studies outside medicine. Despite large differences in performance, no variations in self-monitoring across semesters (with the exception of the first semester) were observed

Diagnostic error increases mortality and length of hospital stay in patients presenting through the emergency room

Background: Diagnostic errors occur frequently, especially in the emergency room. Estimates about the consequences of diagnostic error vary widely and little is known about the factors predicting error. Our objectives thus was to determine the rate of discrepancy between diagnoses at hospital admission and discharge in patients presenting through the emergency room, the discrepancies’ consequences, and factors predicting them. Methods: Prospective observational clinical study combined with a survey in a University-affiliated tertiary care hospital. Patients’ hospital discharge diagnosis was compared with the diagnosis at hospital admittance through the emergency room and classified as similar or discrepant according to a predefined scheme by two independent expert raters. Generalized linear mixed-effects models were used to estimate the effect of diagnostic discrepancy on mortality and length of hospital stay and to determine whether characteristics of patients, diagnosing physicians, and context predicted diagnostic discrepancy. Results: 755 consecutive patients (322 [42.7%] female; mean age 65.14 years) were included. The discharge diagnosis differed substantially from the admittance diagnosis in 12.3% of cases. Diagnostic discrepancy was associated with a longer hospital stay (mean 10.29 vs. 6.90 days; Cohen’s d 0.47; 95% confidence interval 0.26 to 0.70; P = 0.002) and increased patient mortality (8 (8.60%) vs. 25(3.78%); OR 2.40; 95% CI 1.05 to 5.5 P = 0.038). A factor available at admittance that predicted diagnostic discrepancy was the diagnosing physician’s assessment that the patient presented atypically for the diagnosis assigned (OR 3.04; 95% CI 1.33–6.96; P = 0.009). Conclusions: Diagnostic discrepancies are a relevant healthcare problem in patients admitted through the emergency room because they occur in every ninth patient and are associated with increased in-hospital mortality. Discrepancies are not readily predictable by fixed patient or physician characteristics; attention should focus on context

Differential diagnosis checklists reduce diagnostic error differentially: a randomized experiment

Introduction Wrong and missed diagnoses contribute substantially to medical error. Can a prompt to generate alternative diagnoses (prompt) or a differential diagnosis checklist (DDXC) increase diagnostic accuracy? How do these interventions affect the diagnostic process and self-monitoring? Methods Advanced medical students (N = 90) were randomly assigned to one of four conditions to complete six computer-based patient cases: group 1 (prompt) was instructed to write down all diagnoses they considered while acquiring diagnostic test results and to finally rank them. Groups 2 and 3 received the same instruction plus a list of 17 differential diagnoses for the chief complaint of the patient. For half of the cases, the DDXC contained the correct diagnosis (DDXC+), and for the other half, it did not (DDXC−; counterbalanced). Group 4 (control) was only instructed to indicate their final diagnosis. Mixed-effects models were used to analyse results. Results Students using a DDXC that contained the correct diagnosis had better diagnostic accuracy, mean (standard deviation), 0.75 (0.44), compared to controls without a checklist, 0.49 (0.50), P < 0.001, but those using a DDXC that did not contain the correct diagnosis did slightly worse, 0.43 (0.50), P = 0.602. The number and relevance of diagnostic tests acquired were not affected by condition, nor was self-monitoring. However, participants spent more time on a case in the DDXC−, 4:20 min (2:36), P ≤ 0.001, and DDXC+ condition, 3:52 min (2:09), than in the control condition, 2:59 min (1:44), P ≤ 0.001. Discussion Being provided a list of possible diagnoses improves diagnostic accuracy compared with a prompt to create a differential diagnosis list, if the provided list contains the correct diagnosis. However, being provided a diagnosis list without the correct diagnosis did not improve and might have slightly reduced diagnostic accuracy. Interventions neither affected information gathering nor self-monitoring

Harnessing the wisdom of crowds can improve guideline compliance of antibiotic prescribers and support antimicrobial stewardship

open access articleAntibiotic overprescribing is a global challenge contributing to rising levels of antibiotic resistance and mortality. We test a novel approach to antibiotic stewardship. Capitalising on the concept of “wisdom of crowds”, which states that a group’s collective judgement often outperforms the average individual, we test whether pooling treatment durations recommended by different prescribers can improve antibiotic prescribing. Using international survey data from 787 expert antibiotic prescribers, we run computer simulations to test the performance of the wisdom of crowds by comparing three data aggregation rules across different clinical cases and group sizes. We also identify patterns of prescribing bias in recommendations about antibiotic treatment durations to quantify current levels of overprescribing. Our results suggest that pooling the treatment recommendations (using the median) could improve guideline compliance in groups of three or more prescribers. Implications for antibiotic stewardship and the general improvement of medical decision making are discussed. Clinical applicability is likely to be greatest in the context of hospital ward rounds and larger, multidisciplinary team meetings, where complex patient cases are discussed and existing guidelines provide limited guidance

“I’ve Just Been Pretending I Can See This Stuff!”: Group Member Voice in Decision Making with a Hidden Profile

This research seeks to expand our knowledge of what underlies group performance in Hidden Profile decision tasks, adopting a mixed methods approach. We created a new mental simulation intervention designed to improve group decision outcomes and information exchange and tested it across two studies. We supplemented our quantitative statistical analysis with Thematic Analysis, to explore and better understand the motivations and utterances of individual group members, which we contend are key to increasing understanding of the challenges operating at individual and group levels in Hidden Profile decision tasks. Much group decision-making research uses quantitative methodologies, searching for causal explanations of why things happen as they do in group processes. As a subset of this area, existent Hidden Profile research is centred in the quantitative domain. Yet qualitative research can improve the understanding of group phenomena, such as communication style, which is important in groups’ decision-making. To our knowledge, no Hidden Profile research has taken a similar approach, so this paper makes a unique contribution. Results indicated the mental simulation had a positive effect on information exchange and decision quality in a Hidden Profile hiring task

When Does Diversity Trump Ability (and Vice Versa) in Group Decision Making? A Simulation Study

It is often unclear which factor plays a more critical role in determining a group's performance: the diversity among members of the group or their individual abilities. In this study, we addressed this “diversity vs. ability” issue in a decision-making task. We conducted three simulation studies in which we manipulated agents' individual ability (or accuracy, in the context of our investigation) and group diversity by varying (1) the heuristics agents used to search task-relevant information (i.e., cues); (2) the size of their groups; (3) how much they had learned about a good cue search order; and (4) the magnitude of errors in the information they searched. In each study, we found that a manipulation reducing agents' individual accuracy simultaneously increased their group's diversity, leading to a conflict between the two. These conflicts enabled us to identify certain conditions under which diversity trumps individual accuracy, and vice versa. Specifically, we found that individual accuracy is more important in task environments in which cues differ greatly in the quality of their information, and diversity matters more when such differences are relatively small. Changing the size of a group and the amount of learning by an agent had a limited impact on this general effect of task environment. Furthermore, we found that a group achieves its highest accuracy when there is an intermediate amount of errors in the cue information, regardless of the environment and the heuristic used, an effect that we believe has not been previously reported and warrants further investigation

Evaluation of Colloids and Activation Agents for Determination of Melamine Using UV-SERS

UV-SERS measurements offer a great potential for environmental or food (detection of food contaminats) analytics. Here, the UV-SERS enhancement potential of various kinds of metal colloids, such as Pd, Pt, Au, Ag, Au-Ag core-shell, and Ag-Au core-shell with different shapes and sizes, were studied using melamine as a test molecule. The influence of different activation (KF, KCl, KBr, K 2SO 4) agents onto the SERS activity of the nanomaterials was investigated, showing that the combination of a particular nanoparticle with a special activation agent is extremely crucial for the observed SERS enhancement. In particular, the size dependence of spherical nanoparticles of one particular metal on the activator has been exploited. By doing so, it could be shown that the SERS enhancement increases or decreases for increasing or decreasing size of a nanoparticle, respectively. Overall, the presented results demonstrate the necessity to adjust the nanoparticle size and the activation agent for different experiments in order to achieve the best possible UV-SERS results